Multifingered Grasping Based on Multimodal Reinforcement Learning

In this work, we tackle the challenging problem of grasping novel objects using a high-DoF anthropomorphic hand-arm system. Combining fingertip tactile sensing, joint torques and proprioception, multimodal agent is trained in simulation to learn finger motions determine when lift an object. Binary contact information level-based simplify transferring learned model real robot. To reduce exploration space, first generate postural synergies by collecting dataset covering various grasp types principal component analysis. Curriculum learning further applied adjust randomize initial object pose based on training performance. Simulation robot experiments with dedicated poses show that our method outperforms two baseline models success rate both for seen unseen objects. This approach serves as fundamental technology complex in-hand manipulations multi-sensory